Skip to content

Split Dockerfile into parallel build groups for GitHub Actions#23

Merged
hsbt merged 8 commits intomasterfrom
split-image-build
Mar 4, 2026
Merged

Split Dockerfile into parallel build groups for GitHub Actions#23
hsbt merged 8 commits intomasterfrom
split-image-build

Conversation

@hsbt
Copy link
Copy Markdown
Member

@hsbt hsbt commented Mar 4, 2026

This reduces CI build time from ~3 hours (sequential) to ~30-45 minutes (parallel groups).

This reduces CI build time from ~3 hours (sequential) to ~30-45
minutes (parallel groups).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 4, 2026 05:24
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the Docker multi-stage build into multiple named “group” targets so GitHub Actions can build/cache them in parallel, then assembles a final runtime image using cached outputs to reduce CI wall-clock time.

Changes:

  • Split the Dockerfile into builder-buster/builder-bullseye base stages plus group-* build targets and a final aggregation stage.
  • Add per-target stripping/cleanup and aggregate outputs from each group into the final image.
  • Introduce a GitHub Actions workflow that builds each group-* target in a matrix and then builds/pushes the final image using registry-backed BuildKit cache.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.

File Description
Dockerfile Introduces per-version build groups as separate stages and aggregates their artifacts into the final image.
.github/workflows/build.yml Adds a parallelized CI build that caches each group target separately and then builds/pushes the final image.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread Dockerfile
Comment on lines 16 to 17
RUN echo "deb http://archive.debian.org/debian/ buster main contrib non-free" > /etc/apt/sources.list && \
echo "deb http://archive.debian.org/debian-security/ buster/updates main contrib non-free" >> /etc/apt/sources.list && \
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stage uses archive.debian.org (Buster) and disables Acquire::Check-Valid-Until for the apt-get update here, but there is another apt-get update later in the same stage (after adding i386 + deb-src) that will still hit expired Valid-Until unless the check is also disabled/persisted. To avoid flaky/failing builds, apply Check-Valid-Until=false to the later update as well (or add an apt conf entry so it applies to all updates in this stage).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The find … -print0 | xargs -0 strip pipeline can fail when no files match the file | grep -q "not stripped" filter (GNU xargs will still invoke strip with no args). Consider avoiding xargs here (e.g., use find … -exec strip … {} +) or add --no-run-if-empty/-r to xargs; also quoting the filename in the file call would be safer.

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: this find … | xargs strip can invoke strip with no args if the file | grep filter selects nothing. Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r (and quote the filename passed to file).

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: if the file | grep -q "not stripped" test matches nothing, xargs may still run strip with no arguments and fail the build. Use find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: xargs strip may be executed with no operands when the preceding find prints nothing, causing the layer to fail. Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: consider avoiding … -print0 | xargs -0 strip because xargs can run strip with no args if no files are selected. Use find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: xargs strip can fail if the file | grep predicate filters everything out (xargs may still invoke strip with no inputs). Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.
Comment thread Dockerfile
Comment on lines 193 to 195
gcc \
rdfind \
${system_ruby} \
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rdfind is now installed in the final runtime image only to run the hardlink dedup step. If the runtime image size/attack surface matters, consider running rdfind in an intermediate build stage (after aggregating the group outputs) and then copying the resulting /build-all-ruby into the final stage, or purging rdfind after the rdfind run within the same layer.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 4, 2026

@hsbt I've opened a new pull request, #24, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 4, 2026

@hsbt I've opened a new pull request, #25, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 4 commits March 4, 2026 05:34
…er-buster

Co-authored-by: hsbt <12301+hsbt@users.noreply.github.com>
Fix expired Valid-Until bypass missing on second apt-get update in builder-buster
… image

Co-authored-by: hsbt <12301+hsbt@users.noreply.github.com>
Move rdfind dedup to intermediate aggregator stage, remove from final runtime image
@hsbt
Copy link
Copy Markdown
Member Author

hsbt commented Mar 4, 2026

Let's try to this.

@hsbt hsbt merged commit 89036eb into master Mar 4, 2026
@hsbt hsbt deleted the split-image-build branch March 4, 2026 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants